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(54) Method and apparatus for the display of video images. 



(57) Video images are displayed from a sequence 
of frames or lines of video signals from a pre- 
recorded source of frames 335 or lines, respect- 
ively. A direction that a user is facing is 
determined by a directional device 200 and a 
portion of the sequence of frames or lines are 
stored based on the direction determined by the 
directional device. The stored portion is read 
from memory and displayed for the user. 
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Technical Field of the Invention 

This invention relates to the recording and play- 
back and display of video images and more particu- 
larly to the recording and display of video images for 
a virtual reality environment. 

Background of the Invention 

Recent advances in hardware and software have 
permitted an increased awareness and development 
effort in the area of interactive "synthesized" environ- 
ments. The goal of this development effort is to pro- 
vide a synthesized environment which is so realistic 
as to be virtually indistinguishable from the absolute 
or actual reality. The application of these inner-relat- 
ed synthesized realities are diverse: vehicle simula- 
tion, preconstruction, architectural review and walk- 
through, video game-like entertainment, molecular 
mapping and engineering, microsurgery, communi- 
cation and educational applications. 

The use of the vehicle simulator which may sim- 
ulate the operation of vehicles, such as race cars and 
aircraft, for entertainment is becoming widespread. 
Also, the apparatus which simulates the operation of 
vehicles are increasingly being used as training de- 
vices for government and industry vehicle operators. 
Such apparatus may be programmed to realistically 
simulate the motion of the vehicle through two or 
three-dimensional space which also can provide rel- 
atively realistic simulation of the control of the vehi- 
cle. Importantly, it is safe and more cost effective for 
a person to be entertained or trained on a video sim- 
ulator, as compared with the real vehicle. However, 
one area that has been substantially overlooked is a 
method and apparatus for recording and displaying 
real images that can be used with a virtual reality sys- 
tem. 

Summary of the Invention 

The present invention provides a virtual reality 
environment using real images that have been re- 
corded and are displayed at a reduced cost as com- 
pared with computer generated or other high perfor- 
mance systems. The present invention provides a 
method and apparatus for collecting, recording and 
displaying this video information from real images 
that have been prerecorded. Thus, the present inven- 
tion provides a more realistic virtual reality environ- 
ment since real images which have been prerecorded 
are displayed in accordance with the movement of the 
user. For example, images of a roller coaster ride, a 
car race, a balloon ride, a Grand Canyon white water 
expedition can be prerecorded to provide a thrill seek- 
ing activity. These prerecorded images can be dis- 
played, for example, in a full-head device, for exam- 
ple a headset having N display screens approximat- 



ing, for example 120 degrees of a field of view. Includ- 
ed within the full-head device is a menu for selecting 
any of these "thrill seeking experience." Once the 
user has selected from the menu, the user can real- 
5 istically view the roller coaster ride, the car race, the 
balloon ride, or the grand-canyon white water expedi- 
tion. However, since the present invention simultane- 
ously records video of the activity that includes a full 
360 degrees, the user is not limited to merely sitting 

10 passively and viewing the prerecorded images. The 
user may interact with the presentation in that the 
user is able to view any object which may interest the 
user. For example, while viewing the prerecorded im- 
ages, the user may notice off to one side of his field 

15 of vision an interesting object, and the user may 
choose to view this object more closely by viewing the 
object in the center of his field of vision. The user is 
able to turn his head; the prerecorded images change 
and this object is placed into the center of his field of 

20 vision, providing a realistic experience. Additionally, if 
the user desires to view a scene that is occurring be- 
hind his head, the viewer may turn his head 180 de- 
grees and realistically view this scene which had been 
previously behind his head. The present invention 

25 provides a method and apparatus for recording and 
displaying images, for example, on a VCR, providing 
a full 360 degree field of vision to a user through ac- 
tual video and taking into account the movement of 
the user while being displayed. 

30 The present invention includes an apparatus to 

display video images for a user, including circuitry for 
generating a sequence of frames of video signals 
from a prerecorded source of frames, the sequence 
of frames corresponding to different points of view of 

35 a scene, circuitry for determining a direction that the 
user is facing by a directional signal indicative of the 
direction that the user is facing, circuit for storing a 
portion of the sequence of frames from the prerecord- 
ed source of the frames in accordance with the direc- 

40 tional signal indicative of the direction that the user is 
facing in a memory, circuitry for reading the memory 
said stored portion of the sequence of frames from 
the prerecorded source of frames in accordance with 
the directional signal indicative of the direction that 

45 the user is facing, a display coupled to the memory for 
displaying said stored portion of the sequence of 
frames. 

Description of the Drawings 

50 

For a more complete understanding of the pres- 
ent invention, and the advantages thereof, reference 
is now made to the following description taken in con- 
junction with the accompanying drawings in which: 
55 Fig. 1 is a circuit diagram of the video recording 

system of the present invention; 

Fig. 2 is a timing diagram illustrating time interval 

and the frame/line sequence to be recorded. 
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Fig. 3 is a circuit diagram of the present invention 
illustrating a portion of the playback and display 
circuit for frame- by-frame display; 
Fig. 4 is a circuit diagram of the present invention 
illustrating a remaining portion of the playback 5 
and display circuit for frame- by-frame display; 
Fig. 5 is a timing diagram illustrating time inter- 
vals, and the operation of the buffers to control 
the storage and output of image data when the 
field of view of the user is within two cameras. 10 
Fig. 6 is a diagram timing illustrating time interval 
and the operation of the buffers to control the 
storage and output of image data when move- 
ment within the field of view of the user is within 
three cameras. 15 
Fig. 7 is a circuit diagram illustrating a portion of 
a display circuit for line-by-line display; 
Fig. 8 is a circuit diagram illustrating the remain- 
ing portion of the display circuit for line-by-line 
display; and 20 
Fig. 9 is a perspective diagram of a helmet device 
with a flux-gate compass. 

Detailed Description of the Invention 

25 

The present invention and its advantages are 
best understood by referring to Figs. 1-9 of the draw- 
ings, like numerals being used for like and corre- 
sponding parts of the various drawings. 

As illustrated by Fig. 1 , a video collection system 30 
10 is Illustrated. The video collection system 10 is for 
collecting and recording a sequence of images for 
generating a virtual reality environment. The present 
invention collects a series of images that may repre- 
sent a full 360 degree field of vision. The 360 degree 35 
field of view is obtained by a plurality of cameras 1 00 
positioned so that the field of vision of all of the cam- 
eras receives the images that include the 360 degree 
field of vision. The cameras may be equally distanced 
positioned along radius of a circle. The angles be- 40 
tween these radius may be equal so that each of the 
cameras receive an equal portion of the 360 degree 
field of vision. The images from these cameras are 
converted to video signals and selected so a respec- 
tive sequence of images from each camera is pro- 45 
duced and this sequence is subsequently recorded in 
a recording device, for example, a VCR. The cameras 
continuously receives the images as a transporting 
device of the cameras traverse a predetermined path 
at a predetermined rate, for example, along a path for so 
a roller coaster, a highway or a hallway. In this man- 
ner, the images and the corresponding video signals 
change as the scene along the path changes. 

These video signals are recorded for playback 
and displayed at a later time. When these sequence 55 
of video signals are received during playback of the 
VCR, a portion of the video signals are selected to be 
stored in a memory and subsequently displayed on a 



display screen. This selected video signals are deter- 
mined by the number of cameras, and the direction 
that the user is facing as the VCR is playing back the 
images. The number of cameras is selected such that 
the 360 degree full view of vision is achieved. For ex- 
ample, two cameras may be sufficient if each camera 
has a field of vision of 180 degrees. It is with this 
scope of invention that the field of vision of each cam- 
era overlap. During playback and display, not all of 
the video signals corresponding to the image re- 
ceived by the cameras are to be displayed at one 
time. For example, the image at the back of the head 
of the user are not displayed until the user turns 
his/her head. However, generally in order to provide 
a realistic display, the images from at least one cam- 
era is required to be displayed on the display. This re- 
sults in a problem in that the video signal correspond- 
ing to multiple images should be combined into the 
display. This problem is especially evident while the 
user is looking at a position that is within the field of 
view of two cameras. Since the video signals for a 
moving picture includes a number of lines that are in- 
cluded with a frame, each of the lines of the display 
may include portions of several lines from several 
cameras. There is an additional problem in merging 
the lines of the frames to be displayed caused by mul- 
tiple cameras, and additional problems result from 
the change of the contribution of the lines from the va- 
rious cameras to the lines to be displayed. This addi- 
tional problem is caused by the movement of the user. 
As the user moves, the contribution of each camera 
to the line to be displayed changes. As the movement 
of the user continues, the contribution of one camera 
may be completely eliminated and a new camera may 
begin to contribute to the line to be displayed. Thus, 
the particular combination of cameras contributing to 
the lines of frame to be displayed may change in ac- 
cordance with the movement of the user. Moreover, 
while the actual camera contribution that contributes 
to the lines or frames of the display remains constant 
as the user moves, the percentage of a line from a 
camera contributing to the line or frame to be dis- 
played may change continuously in accordance with 
the movement of the user. In order to solve the above 
mentioned problems, the line or frames of the camera 
contributing to the display may be continuously stor- 
ed so as the stored lines or frames may be continu- 
ously read out the storage and displayed. One meth- 
od of achieving the above mentioned goals is to con- 
tinuously store each frame or line that may contribute 
to the line orf rame to be displayed. Additionally, since 
a sequence of video signals have been recorded and 
played back, the video signals not contributing to the 
frames or lines to be displayed may be ignored. How- 
ever, the display must be continuously supplied with 
video signals in order not to produce any flicker or dis- 
continues on the display. This requires that lines or 
frames of the video signals be retained in storage so 
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as to provide a source of video signals as the noncon- 
tributing video signals are being played from the 
VCR. Furthermore, the relative contribution of video 
signals displayed may be altered as the user moves. 

Video signals from two cameras may contribute 
to the lines or frames of the image displayed. Further- 
more, cameras may be employed to record the 360 
degree full field of view. However, it is to be under- 
stood that any number of image devices may be em- 
ployed to record the 360 degree full field view. Addi- 
tionally, more than two image devices may be em- 
ployed to contribute at any one time to the lines or 
frames to be displayed by employing additional mem- 
ory and control circuits. 

The image generating devices 100 may be, for 
example, a series of cameras, which are gen-locked 
together so that the horizontal and vertical syncs are 
respectively synthesized. The image generating de- 
vices 100 obtain the full 360 field of vision by obtain- 
ing video images along a path. The video images ob- 
tained from the image generating devices 100 are 
converted into video signals by the image generating 
devices 100 and output to the video multiplexing de- 
vice 110. The image generating devices 100 are con- 
trolled to generate a sequence of images. Each of the 
i mage generating devices 1 00 output a frame of a vid- 
eosignal and afterthe frame is complete, the next im- 
age generating devices begins to output a video sig- 
nal. This sequence of video signals are generated by 
the sequential operation of the image generating de- 
vice 100. Advantageously, the image generating de- 
vices 1 00 are arranged to record the full 360 degree 
field of vision advantageously, the field of vision of the 
image generating device 1 00 is limited such that each 
of the image generating devices 100 is arranged to 
limit the images received to N divided by 360 degrees 
where N is the total number of gen-locked cameras. 
Each of the image generating devices may limit the 
field of view of the image generating device by optics 
or alternately the field of view may be limited by soft- 
ware. The video signals from each of the image gen- 
erating devices 100 are input to switching devices 
112, which are controlled by the switch controller 114. 
Each of the switching devices 112 selects the video 
signals from the respective image generating device 
1 00 sequentially. The video images from image gen- 
erating device 100, is output to the multiplexor device 
110. The video multiplexor 110 outputs the sequence 
of image signals to the recording device 170. 

Fig. 1 additionally illustrates sync generator 120 
to receive the video signals from the image generat- 
ing devices 100 and to output horizontal sync signals 
and vertical sync signals to frame/line mode switch 
130. Frame/line mode switch 130 controls the ad- 
dress counter 140 in accordance with whether or not 
frames or lines are desired or selected to be recorded 
in sequence by the recording device 170. 

The frame/line mode switch 130 is a switch that 



the user switches between a line position to record a 
sequence of lines from each of the image generating 
devices 100 and a frame position to record a se- 
quence of frames from each of image generating de- 

5 vices 100. 

If lines are desired to be recorded in sequence, 
the frame/line mode switch 130 connects terminals 
1 32 and 1 34 to the address cou nter 1 40 and the clear 
device 150, respectively. If frames are desired to be 

10 recorded in sequence, the frame/line mode switch 
130 connects terminal 136 and terminal 138 to ad- 
dress counter 140 and clear device 150, respectively. 
In accordance with the position of the frame/line 
mode switch 130, the address counter 140 receives 

15 either the horizontal sync signals or the vertical sync 
signals to control the switch controller 114. As the 
sync signals are input to the address counter 140, the 
address counter 140 increases the count signal, out- 
put to switching device 112. As each count signal is 

20 input to the switch controller 114, the address counter 
140 continues to output the count signals until the se- 
quence of either line frames has been completely re- 
corded by the recording device 170. In order to reset 
the address count 140, the clear device 150 outputs 

25 a clear signal to the address counter 140. The clear 
signal is output while the frame/line mode switch 1 30 
is positioned in the line position and during a vertical 
sync signal and during a reset signal output from the 
counter reset device 160. The clear signal is output 

30 while the frame/line mode switch 1 30 is positioned in 
the frame position only during a reset signal from the 
counter reset device 160. The reset signal is output in 
accordance with the switch 1 80. The switch 1 80 is se- 
lected by the user to select the number of cameras 

35 and the number of frames/lines in the sequence of 
video signals to output a binary representation of the 
member of cameras used when the count signal has 
reached or is equal to the number of image generat- 
ing devices 100. For example when six image gener- 

40 ating devices are to be selected to input video signals 
and when the count signal has reached six, the coun- 
ter reset device 160 outputs a logical "one". As the 
vertical sync signal is input to the address counter 
140, the clear device 150 outputs a logical "one" to 

45 clear and reset the address counter to zero. 

The same sequence is followed while the 
frame/line mode switch 130 is positioned in the line 
position. 

The switch controller 114 controls the switching 
so devices 112, so that either lines or complete frames 
from the image generating devices 100 are connect- 
ed to bus 1 1 6 to output the video images to be record- 
ed by the recording device 170. As the count signals 
are input to the switch controller 114, the count sig- 
55 nals control the switch controller 114 to select one of 
the switch devices 112. Although the switch devices 
112 could be selected in any order, the switch device 
may be selected sequentially from camera 1 to cam- 
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era 6, in order of the address count signal. As the 
switch device 11 2 is selected by the switching control- 
ler 114, the video signals output from the image gen- 
erating device 100 connected to the selected switch 
device 112 is output to the VCR 170. 5 

The image generating devices 100 generate vid- 
eo signals from image input. The video signals are in- 
put to switching device 112, and the switch controller 
114 sequentially controls the switching device 112 so 
that the video signals are output to bus 116 sequen- 10 
tially. For example, Fig. 2 illustrates the time intervals 
and the output video signals from the image generat- 
ing devices. The horizontal and vertical sync signals 
are transmitted to the frame/line mode switch 130. 
Either the horizontal sync signal or the vertical sync 15 
signal is transmitted to the address counter 140. The 
address counter sequentially addresses the switch 
controller 1 14 as the horizontal sync signal or the vert- 
ical sync signal is input to the address counter 140. 
The address counter 140 outputs the count signal as 20 
the horizontal sync or vertical sync signal is input. The 
count signals are also input to the counter reset de- 
vice 160 through the switch 180, which has been set 
to the desired number of image generating devices 
100. The counter reset device 160, for example an 25 
"and" circuit outputs a signal when the last in the se- 
quence of the image generating devices 100 has 
been recorded for example six. The output of the 
counter reset device 160 is input to the clear device 
1 50, for example an "or" device. The counter reset de- 30 
vice 160 is connected to the clear input of the address 
counter 140 so that the first image generating device 
100 is subsequentially addressed. 

Figs. 3 and 4 illustrate the playback system of the 
present invention for frame-by-frame playback. The 35 
recording device 170 having been previously record- 
ed is set in a playback mode to supply a sequence of 
frames as illustrated in Fig. 2. For example, frames 
1 , 2, 3, 4, 5 and 6 have been previously recorded and 
are output to the sync generator 210 and the connec- 40 
tor circuit 250. The directional device 200 positioned 
on a helmet 203 to be worn by the user to indicate the 
direction of the user of the virtual reality system out- 
puts a directional signal indicating the direction of the 
user. The buffer circuit 230 connects the output of the 45 
directional device 200 to the processor 220 and to the 
first read counting device 270, the second read count- 
ing device 280 and the read line counting device 260 
from the processor 220 by a bus. The sync generator 
circuit 21 0 inputs the output of the recording device so 
170 and inputs the sequence of frames as video sig- 
nals. The output of the sync generator 21 0 includes a 
horizontal sync signal, a vertical sync signal and a 
frame indicator signal. The horizontal sync signal, the 
vertical sync signal and the frame indicator signal are 55 
input to the processor 220 to generate a control signal 
to control the interface circuit 240. 

The processor 220 outputs to the write counting 



device 290 the number of pixels in a line of a video sig- 
nal. The interface circuit 240 employs the write pixel 
counting device 290 in order to write both the video 
signal of the first image generating device 100 and 
the video signal of the second image generating de- 
vice 100 since the control of the memory is preformed 
sequentially. After the interface circuit 240 employs 
the write pixel counter counting device 290 for storage 
of the video signal of the first image generating de- 
vice, the interface circuit 240 employs write pixel 
counting device 290 for storage video signal of the 
second image generating device. Additionally, inter- 
face circuit 240 controls the write line counting device 
295 by outputting to the write line counting device 295 
a signal to begin a new line at the correct time as in- 
dicated by the horizontal sync signal. 

The processor 220 controls the read line counting 
device 260 by outputting to the read line counting de- 
vice 260 a count of the number of lines in a frame. The 
processor 220 controls the first read pixel counting 
device 270 by outputting to the first read pixel count- 
ing device 270 the number of pixels from the first vid- 
eo signals of one of the image generating devices 1 00 
in that line to be displayed. The processor 220 con- 
trols the second read pixel counting device 280 by 
outputting to the second read pixel counting device 
280 the number of pixels remaining in the line of the 
second video signal again in accordance with direc- 
tional signal of the directional device 200. The inter- 
face circuit 240 controls the read line counting device 
260 by outputting a signal to begin the next line at the 
correct time. As indicated by the horizontal sync sig- 
nal the interface circuit 240 receives, from the first 
counting device 270, a first count down signal when 
the first read counting device 270 has counted the 
number of pixels which have been inputted and cor- 
responding to the video signal of the first image gen- 
erating device 100 in a line. Additionally, the proces- 
sor 220 receives a second count down signal from the 
second read counting device 280 when the second 
read counting device 280 has counted the number of 
pixels corresponding to the video signal. These first 
and second read pixel counting devices 270, 280 pro- 
vide control and timing for the read from memory. Ad- 
ditionally, the interface circuit 240 controls the writing 
to the frame storage circuits 335, 340, 345 and 350. 

The interface circuit 240 controls the first input 
buffer circuit 315, the second input buffer circuit 320, 
the third input buffer circuit 325 and the fourth input 
buffer circuit 330. The interface circuit 240 enables 
and disables the input buffer circuits 315, 320, 325 
and 330 by outputting respective enabling signals, 
disabling signals and as a consequence of these sig- 
nals the interface circuit 240 additionally controls first 
frame storage circuit 335, the second frame storage 
circuit 345, the third frame storage circuit 350 and the 
fourth frame storage circuit 340 so that only one of the 
frame storage circuits 335, 340, 345 and 350 are en- 
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abled to store a frame of the video signals at any given 
time, and the remaining frame storage circuits 335, 
340, 345 and 356 are isolated by the remaining input 
buffer circuits 315, 320, 325 and 330. The interface 
circuit 240 controls the first output buffer circuit 355, 
the second output buffer circuit 360, the third output 
buffer circuit 365 and the fourth output buffer circuit 
370 to isolate the respective frame storage circuit 
335, 345, 350 and 340 by enable and disable signals 
so that these circuits output the stored video signals 
according with the enable signals. The interface cir- 
cuit 240 controls the AID converter circuit 300 and the 
D/A converter circuit 375 to control the in put of the red 
video, green video and blue video signals to the first, 
second, third and fourth input buffer circuits 315, 320, 
325 and 330 to the input of the frame storage circuits 
335, 340, 345, 350. Additionally, the interface circuit 
240 controls the first address device 305 and the sec- 
ond address device 310 so as to select the address 
for either the first frame storage circuit 335 and the 
second frame storage circuit 345 or the fourth frame 
storage circuit 340 and the third frame storage circuit 
350. The read line countjng device 260 receives from 
the processor 220 the count of the total number of 
lines in a frame and interface circuit 240 outputs a 
read line counting signal, to indicate the end of the line 
in order for the next line to be displayed. The first read 
pixel counting device 270 receives (from micropro- 
cessor 220) the number of pixels in the line of the first 
video signal from the image generating device 100 
and outputs the first count down signal when the num- 
ber of pixels have been read from the frame storage 
circuit to indicate that the portion of the line corre- 
sponding to the first video signal has been displayed 
from memory. The same function is performed by the 
second read pixel counting device 280 for the second 
video signal. The second read counting device 280 re- 
ceives (from the processor 220) the number of pixels 
in the line of the second video signal from the image 
generating device 100. The second read counting de- 
vice 280 counts from the input count to output a sec- 
ond count down signal to indicate that the remaining 
pixels corresponding to the second video signal has 
been read from the frame storage circuit The read 
line counting device 260 is controlled by the interface 
circuit 240 to display the next line. At this point the to- 
tal number of pixels for a line for a frame to be dis- 
played has been output from the frame storage cir- 
cuit. 

Referring to Fig. 4, the analog to digital converter 
A/D device 300 converts the analog video signal to 
digital video signals. Although, the circuit of Fig. 4 il- 
lustrates the video channel circuit for a red video sig- 
nal, it is to be understood that an additional video 
channel circuits similar to the circuit illustrated in Fig. 
4 may be provided for the storage of both the green 
video and the blue video. 

The first address device 305 provides addresses 



for either the first frame circuits storage 335 and the 
second frame storage circuit 345 for both reading and 
writing to these storage circuits. These addresses are 
provided for the first address device 305 by either the 

5 first read pixel counting device 270, the second read 
pixel counting device 280, the read line counting de- 
vice 260 or the write pixel counting device 290 and 
write line counting device 295. 

The second address device 31 0 addresses either 

10 the fourth frame storage circuit 340 or the third frame 
storage circuit 350. These address are provided for 
the second address device 310 by the first read pixel 
counting device 270, the second read pixel counting 
device 280, the read line counting device 260, or the 

15 write pixel counting device 290 and the write line 
counting device 295. 

The first input buffer circuit 315 buffers and con- 
trols the data, for example pixels of the video signal, 
to first frame storage circuit 335. The second input 

20 buffer circuit 320 buffers and controls the data for ex- 
ample pixels of the video signal to the second frame 
storage circuit 345. The third input buffer circuit 325 
buffers and controls the data for example pixels of the 
video signal to the third frame storage circuit 350. The 

25 fourth input buffer circuit 330 buffers and controls the 
data for example pixels of the video signals to fourth 
frame circuit 340. These input buffer circuits are con- 
trolled by the enabling signals to enable the input buf- 
fer circuit 315, 320, 325 and 330 from the interface 

30 device 240. 

With six image generating devices 100, the field 
of view presented to the user either is from only one 
image generating device 100 or two image generat- 
ing devices 100. Consequently, a line to be displayed 

35 to the user may include pixel information from either 
one of two these image generating devices 1 00. As a 
consequence, the same number of frame storage cir- 
cuits, namely two frame storage circuits, may be util- 
ized to store the portions of the line corresponding to 

40 these two image generating devices 100. Additional- 
ly, while these two frame storage circuits written to, 
additionally two other frame storage circuits may be 
required to read the previously stored video frame im- 
age while the user is turning his head to provide a dis- 

45 play of the image. The first frame storage circuit 335 
may store a video signal in the form of frames for one 
of two images from the image generating devices 
1 00. The fourth frame storage circuit 340 having pre- 
viously stored video frame i mage now may output vid- 

so eo signals in the form of frames for one of the two im- 
age generating devices 100. Correspondingly, the 
first frame storage circuit 335 and the fourth frame 
storage circuit 340 are alternately read and output the 
video frame image. The second frame storage circuit 

55 345 stores the video frame from another of the image 
generating devices 100, and the third frame storage 
circuit 350 having previously stored a video frame im- 
age from another of the two image generating devic- 
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es 100 may now output the video frame image. Fur- 
thermore, since at any time, only one frame storage 
circuit of the four frame storage circuits (335, 340, 
345, 350) is selected for output to the display 380, out- 
put buffer circuits control the output from the frame 
storage circuits so that only one frame storage circuit 
is selected and output. 

This output of the lines of the video signals con- 
tinue until the entire frame has been output to the dis- 
play 380. As the output progresses, lines of other vid- 
eo signals are being stored within the third and fourth 
frame storage circuits (340, 350). 

When the first and second frame storage circuits 
(335 and 345) have completed the output of lines and 
the third and fourth frame storage circuit 340 and 350 
have completed the storage of the video signals, the 
function of these frame storing circuits are switched 
such that the video signals are output from the third 
and fourth storage circuit 340 and 350 while the first 
and second frame storage circuit 335 and 345 store 
video signals. 

The first output buffer circuit 355 is enabled by 
the interface circuit 240, and a portion of a line of the 
video frame image stored in the first frame storage 
circuit 335 are output to the digital to analog converter 
circuit 375. The second output buffer circuit 360 is 
controlled and enabled by the interface circuit 240 so 
that the remaining portion of the line of the video 
frame image stored in the second frame storage cir- 
cuit 345 are output to the digital analog converter cir- 
cuit 375. The third output buffer circuit 365 is enabled 
by the interface circuitry 240 so that a portion of a line 
of the video frame images stored in the third frame 
storage circuit 350 are output to the digital to analog 
converter circuit 375. The fourth output buffer circuit 
370 is enabled and controlled by the interface circui- 
try 240 so that the remaining portion of the line of the 
video frame image stored in the third frame storage 
circuit 350 is output to the digital to analog converter 
circuit 375. 

The digital to analog converter circuit 375 con- 
verts the video frame images that have been stored 
as video signals and that is output from either the first 
output buffer circuit 355, the second output buffer cir- 
cuit 360, the third output buffer circuit 365 or the 
fourth output buffer circuit 370. The output of the dig- 
ital to analog converter circuit 375 is combined with 
the output of the digital to analog converter circuits of 
the corresponding green and blue channel circuits 
(not shown) to be displayed by the display, for exam- 
ple, a television or a projection screen positioned 
within the helmet of the user. 

As the video signals representing the sequence 
of image from image generating devices 100 are out- 
put from the recording device 170, the sync generator 
210 generates the horizontal syncsignals, the vertical 
sync signals and the frame indicator signals from the 
sequence of images, and these signals are input to 



the processor 220 and the interface circuit 240; at the 
same time, a directional signal from the directional 
device 200 is input to the processor 220, the first read 
counting device 270, the second read counting de- 

5 vice 280 and the read line counting device 260. The 
processor 220 determines the relative percentage of 
contribution of each camera to the line in accordance 
with the directional signal from the directional device 
200. These percentages are transmitted to counting 

w devices 270 and 280. The interface circuit 240 selects 
a frame storing circuit, for example, the first frame 
storage circuit 335 for storage of the video signals of 
the first image generating device 100. The interface 
circuit 240 enables the first input buffer circuit 315 so 

15 that the video signals may be input to the first frame 
storing circuit 335. The video signals are converted to 
digital video signals by the digital to analog converter 
circuit 300 and transmitted to the first frame storage 
circuit 335. The write pixel counting device 290 out- 

20 puts a write counting signal to the first address circuit 
305 to address the storage locations within the first 
frame storage circuit 335 to store the video signals 
from the analog to digital converters circuit 300. At the 
end of the frame, at vertical sync, the first input buffer 

25 circuit 31 5 is disabled and the second input buffer cir- 
cuit 320 is enabled to store the second video signal 
in the second frame storage circuit 345. The write 
counting device 290 is provided with the number of 
pixels in a line for the second video signal, and the 

30 count from the write counting device 290 again sup- 
plies the addresses for storage through the first ad- 
dress circuit 305 for the second frame storage circuit 
345 to store the second video signals. The red video 
signals for the second video signal is input to the sec- 

35 ond frame storage circuit 345 through the analog dig- 
ital converter circuit 300 as the first address circuit 
305 supplies the address for storage of these video 
signals in the second frame storage circuit 345. As the 
user turns his head between the field of view of the 

40 first and second image generating devices 1 00, a dif- 
ferent percentage of the first signal to the second sig- 
nal is generated. The interface circuit 240 switches to 
storing the pixels of the first and second image gen- 
erating device 1 00 into the th ird and fourth frame stor- 

45 age circuit 340 and 350. Thus, the first frame storage 
335 and the second frame storage circuit 245 are now 
ready to have the first and second video signals read 
from these frame storing devices. A count corre- 
sponding to the percentage of the first video signals 

so in a line is input to the first read counting device 270 
by the processor 220, and a count corresponding to 
the percentage of the second video signal in the line 
is input to the second read counting device 280 by the 
processor 220 and in accordance with the directional 

55 device 200. Additionally, a count corresponding to the 
total number of pixels in a line of both the first video 
signal and the second video signal is input to the read 
line counting device 260. 
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As the first read counting device 270 counts 
down, these counts are input to the first address cir- 
cuit 305 to provide addresses for the first frame stor- 
age circuit 335. The interface circuit 240 enables the 
fourth output buffer circuit 370 to output the stored 
pixels of the fourth frame storage circuit 340. Alter the 
first counting device 270 has reached the inputted 
count, the second read counting device 280 begins 
counting and the counts are output to the first ad- 
dress circuit 305 to address the second frame storage 
circuit 345. The interface circuit 240 enables the third 
output buffer circuit 350 to output and display the re- 
maining pixels for the line and the horizontal sync sig- 
nal as the line is completed. This process is continued 
until the entire frame has been output from the col- 
lective storage of the fourth frames storage circuit 
340 and the third frame storage circuit 350. 

Fig. 5 illustrates the timing of buffers that are ac- 
tivated to output and display information during vari- 
ous time periods. Fig. 5 illustrates the sequence 
when the user is not turning his head, but the view is 
changing. 

Fig. 5 illustrates that in time interval 1, the image 
from the first camera of the image generating devices 
1 00 is written to the first frame storage circuit 335 by 
enabling the first input buffer circuit 315. During time 
interval 2, the second video image from the image 
generating devices 100 is written to the second frame 
storage circuit 345 by enabling the second input buf- 
fer circuit 320. While the images for the third-sixth vid- 
eo signals from the image generating devices 100, 
which are not stored since the user is viewing the ob- 
jects within the first and second camera of the image 
generating device, are output from the recording de- 
vice 170, the third and fourth frame storage circuit 
340 and 350 alternately output the previously stored 
first and second video image by activating the third 
output buffer circuit 365 and the fourth output buffer 
circuit 370 during the time intervals 7-12. The func- 
tion of the first and second frame storage circuit 335 
and 345 is to store video signals while the function of 
the third and fourth frame storage circuits 350 and 
360 is to output video signals. The function of the first 
frame storage circuit 335 and the second frame stor- 
age circuit 345 switches with the third and fourth 
frames storage circuits 340 and 350. 

Fig. 6 is similar to Fig. 5 with the additional fea- 
ture during the time intervals 1 -6, the user has turned 
h is head from between the field of view of the first and 
second camera of the image generating devices 100 
to the field of view between the second and third cam- 
eras of the image generating devices. As illustrated 
in Fig. 6, the second and third video signal from the 
second and third camera of the image generating de- 
vices are stored during the eighth and ninth time in- 
terval in the third and fourth frames storage circuit 
340, 350. The first and second frame storage circuit 
335 and 345 store the second and third video signals 



from the second and third camera of the image gen- 
erating devices 100 during the fourteenth and fif- 
teenth time interval. Thus, as the user turns his/her 
head from the field of view of the first and second im- 

5 age generating device to the second and third image 
generating device the video signals stored corre- 
sponding changes. 

Fig. 7 and 8 illustrate the playback system with 
line-by-line storage. 

10 As illustrated in Fig. 7, the processor 221 controls 

the interface circuit 241. The interface circuit 241 ac- 
tivates the first line storage circuit 440, for example a 
FIFO storage circuit, and the second line storage cir- 
cuit 450, again for example a FIFO storage circuit, by 

15 the respective clock signals, for example the FIFO "A" 
clock signals FIFO storage circuit 440 to store a por- 
tion of a line of the first video signal from the image 
generating device 1 00 and FIFO "B" clock signal FIFO 
storage circuit 450 to store the remaining portion of 

20 the line of the second video signals from the image 
generating device 100 respectively as the first video 
signals and the second video signals are respectively 
output from the recording device 170. As other video 
signals are outputfrom the recording device 170, the 

25 first line storage circuit 440 and the second line stor- 
age circuit 450 are inactivated so that this video infor- 
mation is not stored. 

Each color, namely red, green and blue, have a 
separate channel to store the lines from the first and 

30 second video signals. 

The processor 221 outputs to the first pixel count- 
ing device 410 a first count signal corresponding to 
the number of pixels in a line from the first video sig- 
nal and outputs to the second pixel counting device 

35 4 20 a second count signal corresponding to the num- 
ber of pixels remaining in the line for the second video 
signal. As the directional device 200 changes direc- 
tion, the count of pixel counting device 410 and sec- 
ond pixel counting device 420 changes so that the 

40 number of pixels for the line stored in the first line stor- 
age circuit 440 and the remaining number of pixels are 
stored for the line stored in the second line storage cir- 
cuit is constantly changing in accordance with the di- 
rection from the directional device 200. 

45 The pixel counting device 41 0 counts the number 

of pixels of the first video signal and provides a first 
pixel signal to the interface circuitry 241 when the 
count has been reached. The second pixel counting 
circuit counts the number of pixels of the second vid- 

50 eo signal from image generating device 1 00 and pro- 
vides a second pixel signal to the interface circuitry 
241. 

Referring now to Fig. 8, the first line storage cir- 
cuit 440 stores pixels of the first video of the image 
55 generating device in accordance with a control signal 
for example the FIFO "A" clock signal from the inter- 
face circuit 241. Additionally, the first line storage cir- 
cuit 440 reads out a portion of the line of the first video 
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signal, for example pixels, to the digital to analog con- 
verter circuit 375 in accordance with the second pixel 
signal. Likewise, the second line storage circuit 450 
stores the remaining portion of the line of the second 
video signal. Additionally, the second line storage cir- 5 
cuit 450 outputs the remaining portion of the line of 
the second video signal in accordance with the first 
pixel signal from the first pixel counting device 41 0 to 
the digital to analog converter circuit 375. 

In response to the interface device 241 receiving 10 
the horizontal sync signal, the first pixel counting de- 
vice 41 0 counts the number of pixels in a line from the 
first video signal, the first pixel counting device 410 
outputs the first pixel signal to the interface device 
241. When the interface circuit 241 receives the first 15 
pixel signal, the interface device 241 outputs the con- 
trol signal, for example the FIFO "A" clock signal to 
first line storage circuit 440 to activate the circuit 440. 
When the storage in first line storage circuit 440 is 
complete and to activate the second line storage cir- 20 
cuit 450 to begin storage of the second video signals, 
the first pixel signal is output to the interface device 
241. When the first pixel signal is received by the in- 
terface device 241 , the second count signal is output 
to the second pixel counting device 420. The interface 25 
device 241 output a control signal for example FIFO 
"B" clock signal to the second line storage circuit 450 
to activate the circuit 450. When the second pixel 
counting device 420 has counted the number of pixels 
remaining in the line, the second pixel counting de- 30 
vice 240 outputs the second pixel signal to the inter- 
face device 241 to terminate the storage of the sec- 
ond video signal in the second line storage circuit 450. 
As the third, fourth, fifth and sixth video signals are 
output from the recording device 1 70, neither the first 35 
line storage circuit 440, nor the second line storage 
circuit 450 is activated for storage of these video sig- 
nals. 

Since the first line storage circuit 440 and the sec- 
ond line storage circuit 450 have separate read and 40 
write addresses, the first line storage circuit 440 and 
the second line storage circuit 450 can be read out 
continuously through the digital to analogue conver- 
ter circuit 375 so that the output of the digital analog 
converter circuit 375 is displayed on the displayed de- 45 
vice 380. 

Fig. 8 illustrates that the red, green and blue vid- 
eo signals are stored and output from the first line 
storage circuit 440 and the second line storage circuit 
450 respectively to form a composite display 380. The 50 
playback system using the line-by-line storage of Fig. 
7 and 8 provides a less complicated system than the 
playback system of Figs. 3 and 4. 

Other Embodiments 55 

Although the present invention and its advantag- 
es have been described in detail, it should be under- 



stood that various changes, substitutions and altera- 
tions can be made herein without departing from the 
spirit and scope of the invention as defined by the ap- 
pended claims. 



Claims 

1. An apparatus comprising: 

a first circuit for generating a sequence of 
frames of video signals from a prerecorded 
source of frames; 

a second circuit for determining the rela- 
tive orientation of a user and to generate an ori- 
entation signal indicative of said orientation of 
user; 

a memory for storing a portion of said se- 
quence of frames from said prerecorded source 
of said frames in accordance with said orienta- 
tion; 

a third circuit for reading a stored portion 
of said sequence of frames from memory in ac- 
cordance with the orientation signal; 

2. The apparatus of claim 1 further comprising; 

a display coupled to said memory for dis- 
playing said stored portion of said sequence of 
frames. 

3. The apparatus of Claims 1-2, wherein said sec- 
ond circuit further comprises; circuitry to deter- 
mine a first percentage of a first frame of said se- 
quence of frames and a second percentage of a 
second frame of said sequence of frames based 
on said orientation signal. 

4. The apparatus of claims 1-3, wherein said mem- 
ory further comprises; circuitry for storing said 
portion of said sequence of frames in accordance 
with said first and second percentages. 

5. The apparatus of claims 1-4, wherein said mem- 
ory further comprises; circuitry for storing a first 
portion of said portion of said sequence of frames 
in accordance with said first percentage and for 
storing a second portion of said portion of said se- 
quence of frames in accordance with said second 
percentage. 

6. The apparatus of Claim 5, wherein said memory 
further comprises; circuitry for storing said first 
portion and said second portion of said sequence 
of frames in parallel with a previously recorded 
sequence of frames. 

7. The apparatus of Claims 1-6, wherein said sec- 
ond circuit is a flux gate compass. 
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8. The apparatus of Claim 7, wherein said appara- 
tus further comprises a helmet to mount said flux 
gate compass. 

9. A method for generating a video signal, compris- 5 
ing: 

generating a sequence of frames of video 
signals from a prerecorded source of frames; 

determining the relative orientation of a 
user and producing an orientation signal indica- w 
tive of said orientation of said user; 

storing in a memory a portion of said se- 
quence of frames from said prerecorded source 
of said frames in accordance with said orientation 
signal; 15 

reading said portion of said sequence of 
frames from memory in accordance with said ori- 
entation signal; and generating a video output 
signal comprising said stored portion of said se- 
quence of frames. 20 

10. The method of Claim 9, wherein said method fur- 
ther comprises; determining a first percentage of 
a first frame of said sequence of frames and a 
second percentage of a second frame of said se- 25 
quence of frames based on said orientation sig- 
nal. 

11. The method of Claim 1 0, wherein the method fur- 
ther comprises; storing a first portion in accor- 30 
dance with said first percentage and storing a 
second portion in accordance with second per- 
centage. 

12. The method of Claims 9-11 , wherein the method 35 
further includes the step of storing said first por- 
tion in parallel with said second portion. 

13. The method of Claims 9-12, wherein said method 
further comprises; providing a flux gate com- 40 
pass, the flux gate compass being adapted to 
generate a signal indicative of the relative orien- 
tation of said user. 

14. The method of Claims 9-13, further comprising 45 
displaying said stored portion of said sequence of 
frames on a video display device. 
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